1. Field of the Invention
The present invention relates generally to a technique for reproducing a moving image from its encoded data, and more particularly to an image processing method and apparatus for reproducing a moving image from encoded data having scalability.
2. Description of the Related Art
Generally, an image is encoded (compressed) prior to its recording or transmission. As an encoding method, JPEG is widely used for still images and MPEG for moving images (see, for instance, “Textbook of Latest MPEG,” by Hiroshi Fujiwara [editor], ASCII Press, Jul. 1, 2002).
The use of streaming of moving images via a network such as the Internet is increasing these days. Through the streaming, however, a receiving end is sometimes prevented from receiving the entire code of a frame within a predetermined period of time due to heavy traffic or a failure in a communication channel. In the case of an MPEG moving image, the encoded data of each frame has no scalability. Therefore, in the case of 30 frames per second, if the reception and decoding of the entire code of a frame is not completed within 1/30 second, for instance, the frame is dropped. The MPEG moving image includes independently encoded frames and frames encoded by interframe prediction. If any of the independently encoded frames is dropped, a frame using the dropped frame for interframe prediction may also be dropped. In this case, the smoothness of the motion of the moving image is considerably damaged. In the case of reproducing a moving image while reading its encoded data from a local image source, the same sort of frame dropping also occurs to a frame whose encoded data fails to be decoded within a predetermined period of time.
In MPEG, it is also possible to transmit each frame in separate basic and high-order layers. In this case, it is possible to prevent a frame whose high-order layer cannot be received from being dropped by decoding only its basic layer. The image quality of the frame, however, is considerably degraded (“Textbook of Latest MPEG”).
As coding methods to replace MPEG and JPEG, JPEG2000 (ISO/IEC FCD 15444-1) and its extended method, Motion-JPEG2000 (ISO/IEC FCD 15444-3), have attracted attention (see, for instance, “Next-Generation Image Coding Method JPEG2000,” by Yasuyuki Nomizu, Triceps, Feb. 13, 2001). According to Motion-JPEG2000, a plurality of temporally successive still images is treated as the individual frames of a moving image. Each frame is encoded independent of its preceding and succeeding frames by the JPEG2000 algorithm. Accordingly, each frame of the Motion-JPEG2000 moving image is treatable in the same manner as a still image encoded by JPEG2000.
Data encoded by JPEG2000 has scalability, as is described in detail below. Encoded data of an LRCP progression, for instance, has image quality scalability such that image quality can be controlled with the same resolution. Accordingly, by decoding the encoded data successively from the MLB (most significant bit) layer, an image can be reproduced even if the decoding is prevented from reaching the LSB (least significant bit) layer. That is, even if all the code is not decoded, the image can be reproduced by decoding part of the code (this decoding is referred to as partial decoding in this specification) although its image quality is degraded.
It is also possible to provide scalability to the encoded data of each frame of a Motion-JPEG2000 moving image, so that an image can be reproduced by the partial decoding of each frame. Accordingly, in a streaming operation where, for instance, a transmitting end transmits code of an LRCP progression successively from the MSB layer, and a receiving end may reproduce an image by performing partial decoding on a frame that cannot have the code of all its layers received and decoded within a predetermined period of time. In this manner, frame dropping is avoidable. The frame reproduced by partial decoding, however, has lower image quality than a frame reproduced by having its entire code decoded (or full decoding). A moving image including such a frame of low image quality may provide a sense of strong discomfort. That is, in some cases, such a sense of discomfort may be reduced by dropping the frame of low image quality. Motion smoothness may be damaged by frame dropping. However, in the case of a Motion-JPEG2000 moving image, where each frame is encoded independently, even if a frame of low image quality is dropped, there is no need to drop even its preceding and succeeding frames. Therefore, the reproduced moving image is little affected by the dropping of the frame.